Predictor Combination at Test Time — Supplemental Document

Our algorithm is constructed from a geometric intuition of manifold denoising. As an alternative, we present a second interpretation based on the assumption that our references are constructed explicitly as GP predictive distributions, i.e., from a PAC-Bayesian perspective. We follow the PACBayesian analysis by Pentina et al. [10], derived for linear support vector machine (SVM) cases by casting the linear SVM parameter vector into a Gaussian random vector. In Section 3 of the main paper, we extended it to a sample-based, finite-dimensional projection of infinite dimensional GPs. For a given bounded loss function C(·, ·) ∈ [0, 1], the expected error rateRC and the empirical error rate R̂C of a deterministic predictor g are defined as [9, 12]: